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Many ways of estimating software systems’ reliability, or reliability-related quanti- 
ties, have been developed over the past several years. Of particular interest are methods 
that can be used to estimate a software system’s fault content prior to test, or to discrimi- 
nate between components that are fault-prone and those that are not. The results of these 
methods can be used to: 

• More accurately focus scarce fault identification resources on those portions of a 
software system most in need of it. 

• Estimate and forecast the risk of exposure to residual faults in a software system 
during operation, and develop risk and safety criteria to guide the release of a 
software system to fielded use. 

• Estimate the efficiency of test suites in detecting residual faults. 

• Estimate the stability of the software maintenance process. 


There are practical issues associated with implementing these techniques in produc- 
tion development environments. For example, one recently-developed technique of esti- 
mating a software system’s residual fault content relies on measuring its structural evolu- 
tion throughout its development, and relating the measured structural change to the rate at 
which faults are inserted. Three issues must be resolved to make use of this technique: 

• A baseline for measuring the system’s evolution must be established. During im- 
plementation, this can be done in quite a straightforward manner, since source 
code is often placed under the control of a revision control system such as Clear 
Case or CCC Harvest. It is then a simple matter to define a baseline, and take 
structural measurements of each piece of source code checked in to the controlled 
repository with respect to that baseline. However, in the earlier development 
phases, specification and design artifacts are often much less rigorously controlled 
- it is frequently not possible to establish a baseline and identify with any certainty 
subsequent changes to that baseline. 



• Structural measurements must be taken of the software system’s components. 
This is quite straightforward with source code - measurements can be taken 
automatically as part of checking new or modified software components into the 
repository. This is often more difficult with specification and design artifacts. 
Frequently, specifications and design are not in a machine-readable form. Fur- 
thermore, they are often created using notations with incomplete or ambiguous 
syntax and semantics. Finally, a system may include specification and design ar- 
tifacts of more than one type, and these may be incompatible. A standardized 
notation for representing specification and design artifacts would help to resolve 
the last two issues - one candidate for such a notation is the Unified Modeling 
Language (UML). 

• In identifying relationships between measured structural change and the fault in- 
sertion rate, the number of faults discovered and repaired must be counted. This 
means that a set of rules for identifying and counting faults must be developed. 

In this panel discussion, we discuss these and other practical issues to be addressed in 
implementing software reliability measurement techniques in a production development 
environment. The panelists will describe their experiences in implementing measurement 
programs, the intended and actual results, and suggestions for making them more effec- 
tive. 
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